• Population-Level Function: Describes the relationship between \(Y_i\) and \(X_i\) in the population (finite or super).
• Functional Flexibility: Can be non-linear (!).

• To see this property recall

  \[ \begin{align*} \varepsilon_i &= Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i] \quad \implies\\ {\mathbb{E}}[\varepsilon_i {\:\vert\:}X_i] &= {\mathbb{E}}[Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i] {\:\vert\:}X_i] = 0 \end{align*} \]

• also \({\mathbb{E}}[h(X_i) \varepsilon_i] = 0\). (How can we use Law of Iterated Expectations to prove this?)

• To see this property decompose the squared expression:

\[ \begin{align*} (Y_i - g(X_i))^2 &= \left(Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i] + {\mathbb{E}}[Y_i {\:\vert\:}X_i] - g(X_i)\right)^2 \\ &= \left(Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i]\right)^2 + 2\left(Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i]\right)\left({\mathbb{E}}[Y_i {\:\vert\:}X_i] - g(X_i)\right) \\ &\quad + \left({\mathbb{E}}[Y_i {\:\vert\:}X_i] - g(X_i)\right)^2. \end{align*} \]

• Key insight: If we can estimate \({\mathbb{E}}[Y_i {\:\vert\:}T_i]\) or \({\mathbb{E}}[Y_i {\:\vert\:}T_i, X_i]\) well, we can estimate differences in conditional means—which under the right assumptions are causal effects.

• The question becomes: Can regression help us estimate the CEF?

• Note: There is nothing causal in \({\mathbb{E}}[Y_i {\:\vert\:}T_i]\) or \({\mathbb{E}}[Y_i {\:\vert\:}T_i, X_i]\), so we still need identification

  • We can for example rely on strong or conditional ignorability.

• To see this property we can

  • Use decomposition property of CEF to see \({\mathbb{E}}[ X_i (Y_i - {\mathbb{E}}[Y_i {\:\vert\:}X_i]) ] = 0\)

  • Substitute for \({\mathbb{E}}[Y_i {\:\vert\:}X_i] = X_i^{\prime} b\) and solve

• How plausible is this linearity assumption in practice?

• Always true in the simple case where \(T_i\) is a binary treatment indicator: \({\mathbb{E}}[Y_i {\:\vert\:}T_i] = \beta_0 + \beta_1 T_i\).

• Regression can still be used to approximate the CEF:

• Intuition: Even if CEF is not linear we can use regression to approximate it and make substantive conclusions

• To see this we can decompose the squared error function minimized by OLS

\[ \begin{align*} (Y_i - X_i' b)^2 &= \left( (Y_i - {\mathbb{E}}[Y_i | X_i]) + ({\mathbb{E}}[Y_i | X_i] - X_i' b) \right)^2 \\ &= (Y_i - {\mathbb{E}}[Y_i | X_i])^2 + ({\mathbb{E}}[Y_i | X_i] - X_i' b)^2 \\ &\quad + 2 (Y_i - {\mathbb{E}}[Y_i | X_i]) ({\mathbb{E}}[Y_i | X_i] - X_i' b). \end{align*} \]

• The first term doesn’t involve \(b\).
• The last term has an expectation of zero due to the CEF-decomposition property.

• In the case of CEF linear in \(X_i\), OLS provides estimate of \({\mathbb{E}}[Y_i {\:\vert\:}X_i]\) which is (constant) increase in means of \(Y_i\).

• In the case of CEF non-linear in \(X_i\), OLS provides best linear approximation of \({\mathbb{E}}[Y_i {\:\vert\:}X_i]\)

• In this simple case OLS estimator solves the least squares problem:

  \[ (\widehat{\tau}, \widehat{\alpha}) = {\arg\!\min}_{\tau, \alpha} \sum_{i=1}^n \left(Y_i - \alpha - \tau T_i\right)^2 \]

• Coefficient \(\tau\) is algebraically equivalent to the difference in means (\(\tau_{DiM}\)):

  \[ \widehat{\tau} = \bar{Y}_1 - \bar{Y}_0 = \widehat{\tau}_{DiM} \]

• Using switching equation we can show that:

\[ \begin{align*} Y_i &= T_i Y_i(1) + (1 - T_i) Y_i(0) \\ &= Y_i(0) + T_i ( Y_i(1) - Y_i(0) ) \quad\text{($\because$ distribute)}\\ &= Y_i(0) + \tau_i T_i \quad \text{($\because$ unit treatment definition)}\\ &= {\mathbb{E}}[Y_i(0)] + \tau T_i + ( Y_i(0) - {\mathbb{E}}[Y_i(0)] ) + T_i (\tau_i - \tau) \quad (\because \pm {\mathbb{E}}[Y_i(0)] + \tau T_i)\\ &= {\mathbb{E}}[Y_i(0)] + \tau T_i + (1 - T_i)( Y_i(0) - {\mathbb{E}}[Y_i(0)] ) + T_i (Y_i(1) - {\mathbb{E}}[Y_i(1)]) \quad\text{($\because$ distribute)}\\ &= \alpha + \tau T_i + \eta_i \end{align*} \]

• Linear functional form fully justified by SUTVA assumption alone:

  • Intercept: \(\alpha = {\mathbb{E}}[Y_i(0)]\) (average control outcome).
  • Slope: \(\tau = {\mathbb{E}}[Y_i(1) - Y_i(0)]\) (average treatment effect).
  • Error: deviation of control PO + treatment effect heterogeneity. What is the second interpretation?

\[ \begin{align*} {\mathbb{E}}[\eta_i {\:\vert\:}T_i] &= {\mathbb{E}}[(1 - T_i)( Y_i(0) - {\mathbb{E}}[Y_i(0)] ) + T_i (Y_i(1) - {\mathbb{E}}[Y_i(1)]) {\:\vert\:}T_i] \\ &= (1 - T_i) ({\mathbb{E}}[Y_i(0) {\:\vert\:}T_i] - {\mathbb{E}}[Y_i(0)]) + T_i ({\mathbb{E}}[Y_i(1) {\:\vert\:}T_i] - {\mathbb{E}}[Y_i(1)]) \end{align*} \]

• Does this look familiar? This is selection wrt to \(Y_i(0)\) and \(Y_i(1)\).

• When would this be equal to zero? E.g. under random assignment (strong ignorability)

• Randomization + consistency allow linear model.

  • Does not imply homoskedasticity or normal errors, though!

  • Practical implication: Use heteroskedasticity-robust (HC2) standard errors for inference, e.g. via lm_robust().

• Key insight: If we can identify variables \(X_i\) that explain why some units are treated and others are not, we may still recover causal effects.

• This is the selection on observables framework:

  • Treatment assignment depends on observable characteristics \(X_i\)
  • Once we account for \(X_i\), treatment is “as good as random”
  • We can use regression to adjust for \(X_i\) and estimate causal effects

• Question: What assumptions do we need? And how do we implement this with regression?

• Interpretation: Within groups defined by \(X_i\), treatment is effectively randomized.

  • Units with the same \(X_i\) who are treated vs. untreated are comparable.
  • Any remaining differences in outcomes reflect the causal effect of treatment.

• This is weaker than strong ignorability: we allow \(T_i\) to depend on \(X_i\), just not on \((Y_i(0), Y_i(1))\) after conditioning on \(X_i\).

• Critical requirement: We must observe and correctly measure all confounders \(X_i\).

• Assume constant treatment effects: \(\tau_i = \tau\) for all \(i\).

• Potential outcomes follow: \(f_i(t) = \alpha + \tau t + \eta_i\)

  • \(\alpha\): baseline expected outcome
  • \(\tau\): constant causal effect of treatment
  • \(\eta_i\): individual-specific deviation (captures everything else affecting \(Y_i\))

• The error \(\eta_i\) may depend on covariates. Decompose it as:

\[ \eta_i = X_i^{\prime} \gamma + \nu_i, \]

where \(\gamma\) captures the linear relationship between \(X_i\) and outcomes, and \(\nu_i\) is the residual variation.

• Assumption: \({\mathbb{E}}[\eta_i {\:\vert\:}X_i] = X_i^\prime \gamma\) (linearity in covariates) \(\Rightarrow\) \({\mathbb{E}}[\nu_i {\:\vert\:}X_i] = 0\).

• This looks like a regression equation. But when does OLS identify \(\tau\) causally?

• Under CIA, \(f_i(t) {\mbox{$\perp\!\!\!\perp$}}T_i {\:\vert\:}X_i\), which implies:

\[ \nu_i {\mbox{$\perp\!\!\!\perp$}}T_i {\:\vert\:}X_i \]

• Why? Since \(X_i\) is fixed, only \(\nu_i\) varies in \(\eta_i\). CIA on POs transfers to \(\nu_i\).

• Result: The regression error \(\nu_i\) is:

  1. Uncorrelated with \(X_i\) (by construction of \(\gamma\))
  2. Uncorrelated with \(T_i\) conditional on \(X_i\) (by CIA)

• Therefore, OLS on \(Y_i = \alpha + \tau T_i + X_i^{\prime} \gamma + \nu_i\) yields consistent estimates.

• By CIA, conditioning on \(X_i\) removes selection bias:

\[ {\mathbb{E}}[f_i(t) {\:\vert\:}T_i = t, X_i] = {\mathbb{E}}[f_i(t) {\:\vert\:}X_i] = \alpha + \tau t + X_i^{\prime} \gamma \]

• The causal effect of changing treatment from \((t-v)\) to \(t\):

\[ \begin{align*} {\mathbb{E}}[f_i(t) - f_i(t - v) {\:\vert\:}X_i] &= (\alpha + \tau t + X_i^{\prime} \gamma) - (\alpha + \tau (t - v) + X_i^{\prime} \gamma) \\ &= \tau v \end{align*} \]

• Key observation: The \(X_i^{\prime} \gamma\) terms cancel out!

  • Confounding enters the model additively and is differenced away.
  • The coefficient \(\tau\) represents the causal effect of a unit change in \(T_i\).

• What we achieved:

  • OLS regression of \(Y_i\) on \(T_i\) and \(X_i\) consistently estimates the causal effect \(\tau\)
  • Covariates “absorb” confounding, leaving only causal variation in \(T_i\)

• Critical question: What happens if we omit important confounders from \(X_i\)?

  • If CIA fails because we missed a confounder, our estimate will be biased.
  • This leads us to omitted variable bias…

• To see omitted variable bias we look at what the coefficient on \(T_i\) estimates, \(\frac{{\mathrm{cov}}(Y_i, T_i)}{{\mathbb{V}}(T_i)}\) assuming that the true model should include \(X_i\):

\[ \begin{align*} {\mathrm{cov}}(Y_i, T_i) &= {\mathrm{cov}}(\alpha + \tau T_i + X_i' \gamma + \nu_i,\, T_i) \\ &= \tau {\mathrm{cov}}(T_i, T_i) + {\mathrm{cov}}(X_{1i} \gamma_1 + \ldots + X_{Ki} \gamma_K, T_i) \\ &= \tau {\mathbb{V}}(T_i) + \gamma_1 {\mathrm{cov}}(X_{1i}, T_i) + \ldots + \gamma_K {\mathrm{cov}}(X_{Ki}, T_i) \end{align*} \]

\[ \implies \frac{{\mathrm{cov}}(Y_i, T_i)}{{\mathbb{V}}(T_i)} = \tau + \underbrace{\gamma^{\prime} \delta}_{\text{OVB}} \]

where \(\delta\) are coefficients from regressions of \(X_1, \ldots, X_K\) on \(T_i\).

• By the Frisch–Waugh–Lovell theorem, if we include some of \(X_i\) we will get \(\frac{{\mathrm{cov}}(\tilde{Y}_i, \tilde{T}_i)}{{\mathbb{V}}(\tilde{T}_i)} = \tau + \tilde{\gamma}^{\prime}\tilde{\delta}\), where \(\tilde{\cdot}\) means residualized with respect to included terms from \(X_i\).

• Same holds when we consider the case where we include some controls:

  \[ \text{OVB} = \tilde{\gamma}' \tilde{\delta}. \]

  Everything is just defined in terms of variables that have been residualized with respect to the included controls.

• OVB = confounder impact \(\times\) imbalance (Cinelli and Hazlett 2020).

1. Effect of democratic institutions on growth, estimated via regression of growth on democratic institutions.

2. Effect of exposure to negative advertisements on turnout, estimated via regression of turnout on the number of ads seen.

• Question: What is a possible omitted variable? How will this bias the estimate?

• Pearl develops criteria, which can be directly read off the graph alone.
• Before studying the criteria, we need to define some new concepts.

• Example: \(T \to W \to Y\) is a causal path. \(T \leftarrow Z \to Y\) is a back-door path.

• Intuition: Back-door paths are “alternative explanations” for why \(T\) and \(Y\) might be correlated, even if \(T\) has no causal effect on \(Y\).

• \(T \leftarrow W_1 \leftarrow Z_1 \to Z_3 \to Y\): blocked by \(\{W_1\}\) or \(\{Z_1\}\) (non-colliders).
• \(T \leftarrow W_1 \leftarrow Z_1 \to Z_3 \leftarrow Z_2 \to W_2 \to Y\): blocked by \(\{\emptyset\}\) (\(Z_3\) is a collider).

• Why condition 1? Blocking back-door paths eliminates confounding — the spurious association between \(T\) and \(Y\).

• Why condition 2? Descendants of \(T\) are post-treatment variables. Conditioning on them could block part of the causal effect (if they’re mediators), and/or induce collider/selection bias (if they’re affected by \(T\) and share causes with \(Y\), etc.).

• Result: If \(X\) satisfies the back-door criterion, then it implies conditional ignorability (\(Y_i(t) {\mbox{$\perp\!\!\!\perp$}}T_i {\:\vert\:}X_i\)):

\[ \begin{align*} {\mathbb{E}}[Y_i(t)] &= {\mathbb{E}}_X[{\mathbb{E}}[Y_i {\:\vert\:}T_i = t, X_i]] \implies \\ \implies\ \tau_{ATE} &= {\mathbb{E}}[Y_i(1)] - {\mathbb{E}}[Y_i(0)] = {\mathbb{E}}_X[{\mathbb{E}}[Y_i {\:\vert\:}T_i = 1, X_i] - {\mathbb{E}}[Y_i {\:\vert\:}T_i = 0, X_i]] \end{align*} \]

• Substituting in \(Y\):

  \[ Y = (\beta + \gamma)X + \gamma \varepsilon_Z + \varepsilon_Y \]

• Effect of \(X\) on \(Y\) is \(\beta + \gamma\).

• Controlling for \(Z\), we would estimate an effect of \(\beta\).

• The bias, \(-\gamma\), is the portion of the effect that has been “stolen away” by conditioning on \(Z\).

• Setup with heterogeneous effects:

  • Binary treatment: \(T_i \in \{0,1\}\)
  • Potential outcomes: \(Y_{i}(0)\), \(Y_{i}(1)\)
  • Unit-level treatment effect: \(\tau_i = Y_{i}(1) - Y_{i}(0)\) (varies across \(i\)!)
  • Discrete covariate: \(X_i \in \{x_1, x_2, \ldots, x_L\}\)

• Maintain conditional ignorability (CIA): \(\{ Y_{i}(0), Y_{i}(1) \} {\mbox{$\perp\!\!\!\perp$}}T_i {\:\vert\:}X_i\)

• Goal: Estimate \(\tau_{ATE} = {\mathbb{E}}[\tau_i]\) using regression.

• Key insight: The ATE averages stratum-specific effects \(\tau_x\) by their population shares \({\textrm{Pr}}(X_i = x)\).

• Why saturated?

  • Makes no assumption about functional form of \({\mathbb{E}}[Y_i {\:\vert\:}X_i]\)
  • Each stratum \(x\) gets its own intercept \(\alpha_x\)
  • This is the most flexible linear specification for discrete \(X_i\)

• Question: Does \(\widehat{\tau}\) from this regression estimate \(\tau_{ATE}\)?

• Start from regression anatomy and simplify the covariance:

  \[ \begin{align*} \widehat{\tau} &= \frac{{\mathrm{cov}}(Y_i, \tilde{T}_i)}{{\mathbb{V}}(\tilde{T}_i)} = \frac{{\mathbb{E}}[Y_i \tilde{T}_i] - {\mathbb{E}}[Y_i]\textcolor{#d65d0e}{{\mathbb{E}}[\tilde{T}_i]}}{{\mathbb{E}}[\tilde{T}_i^2]} \\ &= \frac{{\mathbb{E}}[Y_i \tilde{T}_i]}{{\mathbb{E}}[\tilde{T}_i^2]} \quad \text{($\because$ ${\mathbb{E}}[\tilde{T}_i] = 0$)} \end{align*} \]

• Apply law of iterated expectations to the numerator:

  \[ {\mathbb{E}}[Y_i \tilde{T}_i] = {\mathbb{E}}\big[{\mathbb{E}}[Y_i {\:\vert\:}T_i, X_i] \tilde{T}_i\big] \]

  This works because \(\tilde{T}_i\) is not a function of \(T_i\) or \(X_i\).

• So we have: \(\widehat{\tau} = \frac{{\mathbb{E}}\big[{\mathbb{E}}[Y_i {\:\vert\:}T_i, X_i] \tilde{T}_i\big]}{{\mathbb{E}}[\tilde{T}_i^2]}\)

• \(\widehat{\tau}\) aggregates via conditional variance weighting with respect to \(T_i\) instead of just population shares.

• If \(\tau_x\) was constant across \(X_i\), regression recovers ATE, but variance weighting could reduce efficiency (more uncertainty).

• If \(T_i {\mbox{$\perp\!\!\!\perp$}}X_i\), then \(p_x(1-p_x)\) is constant across strata and cancels out, so \(\widehat{\tau}\) reduces to weighting by \({\textrm{Pr}}(X_i = x)\).

• The effective sample is weighted by \(\widehat{w}_i = (T_i - \widehat{{\mathbb{E}}}[T_i | X_i])^2\) (squared residual from regression of \(T_i\) on covariates).

• Even with a representative sample, regression estimates may not aggregate effects in a representative manner. Regression estimates are local to an effective sample.

• Beware of OVB:
  • Avoid “bad controls” that may introduce post-treatment bias or inadvertently open back-door paths.

• Steps to take:
  1. Be explicit about the assumption you need to make to make regression causal.
  2. Use DAGs and the back-door criterion to identify covariates to control for.
  3. Make sure you know how to interpret regression coefficients.
  4. Use simulations and/or Dagitty to validate model assumptions and relationships.